MOHAQ: Multi-Objective Hardware-Aware Quantization of recurrent neural networks

The compression of deep learning models is fundamental importance in deploying such to edge devices. selection parameters can be automated meet changes the hardware platform and application. This article introduces a Multi-Objective Hardware-Aware Quantization (MOHAQ) method, which considers performance inference error as objectives for mixed-precision quantization. proposed method feasibly evaluates candidate solutions large search space by relying on two steps. First, post-training quantization applied fast solution evaluation (inference-only search). Second, we propose ”beacon-based search” retrain selected only use them beacons estimate effect retraining other solutions. We speech recognition TIMIT dataset. Experimental evaluations show that Simple Recurrent Unit (SRU)-based compressed up 8x without any significant increase. On SiLago, found achieve 97% 86% maximum possible speedup energy saving, with minor increase an SRU-based model. Bitfusion, beacon-based reduced gain inference-only Light Gated (LiGRU)-based model 4.9 3.9 percentage points, respectively.